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Abstract 

The concepts of conditional entropy and information between subsys- 
tems of a composite quantum system are generalized to include ar- 
bitrary indirect measurements (POVMs). Some properties of those 
quantities differ from those of their classical counterparts; certain 
equalities and inequalities of classical information theory may be vio- 
lated. 
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In |Levitin 1998[ ILevitin 1999j . concepts of conditional entropy and infor- 
mation were introduced for quantum systems with respect to direct (von 
Neumann) measurements performed over subsystems of a composite quan- 
tum system. In this paper the concepts are generalized to include arbitrary 
indirect measurements (POVMs). The concepts of 'conditional entropy' and 
'information' retain their validity for quantum systems, but their proper- 
ties differ somewhat from those of their classical counterparts; specifically, 
some equalities and inequalities of classical information theory are in general 
violated. 



Consider a composite quantum system consisting of two subsystems A 
and B, the Hilbert space 7i of the system being the tensor product, TLa®'Hb^ 
of the Hilbert spaces of its two subsystems. The state of the system and 
the states of its subsystems are described, respectively, by the joint density 
matrix p(A, B) and the marginal density matrices p(A) and p{B). The joint 
entropy of the system and the marginal entropies of the two subsystems are, 
respectively, H(A, B), H(A), and H(B). 
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Now, let M.a = {M a (A)} be a countable set of self-adjoint nonnegative 
definite operators that form a resolution of the identity (in general, non- 
orthogonal) in Ha, and M.b = {Mb(B)} a similar set of operators in Hb- The 
sets AA.A and A4b correspond to indirect measurements (POVMs) performed 
respectively over the systems A and B. By Naimark's theorem |Naimark~T 940j . 
any POVM is equivalent to a direct (von Neumann) measurement performed 
in an extended Hilbert space. Henceforth we will consider POVMs that 
correspond to measurements of a complete set of variables (represented by 
a complete set of orthogonal one- dimensional projectors) in the extended 
Hilbert space. 1 

Denote by a and (3 the two random variables that are the results of 
measurements and M.b- The probability distributions of a and j3 are 

Pr{a = a} = Tr{p(A)M a (A)}, 

Pr{(3 = b} = Tr{p(B)M b (B)}, (1) 
Pr { a = a , (3 = b} = Tr {p(A, B) \M a (A) ® M b (b)) } . 



Lemma 1. For any choice of A4a and JAb, 

H(A,B)<H(a,(3), 

H{A) < H(a), (2) 
H(B) < H(P). 

Proof. Inequalities (0) follow from Klein's lemma Klein 1 931] by use of 
Naimark's theorem. ■ 

The conditional density matrix of subsystem A given the result of a mea- 
surement performed over subsystem B can be defined for POVMs in a sim- 
ilar way as it is defined for von Neumann measurements, namely, following 
[BaUan 1991] . 

(A\R- h \- Tr ^ MA B)[1(A) ® M 6 (.£?)]} 
P {A\fJ-b)- Tl{p{ABmA) ^ Mb{B)]} , (3) 

where 1(A) is the identity operator in Ha- Note that the denominator in (J3J) 
is just the probability Pr{/9 = b} for (3 to take on value b. 

1 The expression "complete set of variables" is used here in exactly the same meaning as 
"complete set of physical quantities" in Lifshitz 1977] p. 5], namely, as a maximum set of 
simultaneously measurable quantum variables (observables). "Complete set of projectors" 
means that they form a resolution of the identity. Also, since sets Ma and Mb are 
countable, they correspond to measurements of variables with discrete spectrum. 
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Then the conditional entropy of system A, given measurement M. b per- 
formed on B, is 

H(A\(3) = -J2 Tr{p(A B)[I(A) ® M b (B)]} Tr {p(A|/3 = b) \ogp{A\(3 = b)} . 
b 

(4) 

By Klein's lemma, for any a and /3 (i.e., for any measurements Ma and 
Mb performed on A and B), 

H(A\(3) < H(a\0), (5) 

where equality holds iff all p(A\f3 = b) commute and Ma is a von Neumann 
measurement in the basis where all p(A\(3 = b) are diagonal. 

Since conditional entropy is meant to express the uncertainty of the state 
of subsystem A under the constraints imposed on it by the "best" measure- 
ment performed on subsystem B, we propose the following 

Definition 1. The conditional entropy of subsystem A, conditioned by 
subsystem B, is 

H(A\B) = MH(A\(3). (6) 

M B 

The following theorem states that, just as in the classical case, condition- 
ing can only decrease the entropy of a system: 

Theorem 1. 

H{A\B) < H{A). (7) 



Proof. 



H(a\P) < H(a), 



inf H(a\P) < H(a), 

M B 



and 



H(A\B) = inf = b} inf H(a\(3 = b) 

M B ~^ M A 

b 

< inf inf H(a\p) < inf H(a) 

Ma Mb Ma 

= H(A). m 
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It has been pointed out | Yang 2004| that inequality (J7|) follows from the 
nonnegativity of the entropy defect [Tevitin 1969HHolevo 1973] . Indeed, 

H(A)-H(A\B) = 

= inf [-Trp(A) logp(A) + VPr{/3 = b} ■ Tr{p(A\(3 = b) logp(A\(3 = b)}}, 

M B 

b 

where the expression in brackets is a special case of the entropy defect. 

Note that in classical information theory H(a,/3) = H((3) + H{a\f3). 
However, this equality turns into an inequality for quantum systems: 

Theorem 2. 

H(A, B) < H(B) + H(A\B). (8) 

Proof. By definition it suffices to prove that, for any choice of Ais, 

H(A,B)<H(B) + H(A\P). (9) 

It follows from Naimark's theorem that it is sufficient to prove for the 
case when Ai B corresponds to a complete set of orthogonal projectors (a von 
Neumann measurement). 

Let {uib} be the set of orthonormal eigenvectors of the conditional density 
matrix p(A\/3 = b), and {vb} the set of orthonormal vectors corresponding 
to projectors M b (B). Consider a basis in 7i = Ha <S> TCb formed by vectors 
Ui t b ® Vf,. The joint density matrix p(A, B) = | |pib,i'6'I I i n this basis has the 
property that p ih ^ h = \ ib S u >. 

Let us introduce a density matrix p'(A,B) obtained from p(A, B) by 
deleting the off-diagonal elements in the basis described above, namely, 

p'(A, B) = ||p- 6ii / 6 /|| = \\XibSii'S bb '\\. (10) 

For each b, the conditional density matrix is 

p(A\p = b)= ^ 

Also, denote 

p'(B)=Tr A p'(A,B) = \\8 w J2 X ib\\- 

i 

It is readily seen that, since matrices p'(A, B) and p'(B) are diagonal, 

H{A\0) = -Trp'(A, B) \np'(A, B) + Trp'(B) lnp'(B) 

= -Trp(A,B)\np'(A,B) + Trp(S) lnp'(S). ^ ' 
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Consider now the quantum relative entropy (cf. |Schumacher 2000|lVedral 2001p 
between states p(A, B) and p'(A, B), 

H(p(A, B) \\p'(A, B)) = Tip(A, B) In p(A, B) - Trp(A, B) In p'(A, B), (12) 

and similarly that between p(B) and p'{B), 

H(p(B) \\p'(B)) = Trp(B) In p{B) - Trp(S) In p'{B). (13) 

It is well known [Vedral 2001"! p. 13, F2] that partial tracing reduces relative 
entropy; therefore 

H(p(A, B) \\p'(A, B)) > HTr A p(A, B) \\Tr A p'(A, B)) = H(p(B) | \p'(B)), 

which, by (HU), (H2), and (0, yields inequality ©. ■ 

According to classical information theory, the information between the 
outcomes a and (3 of two POVMs M.a and M.b performed on subsystems A 
and B is 

I(a;(3) = H(a) - H(a\/3). (14) 

In the spirit of Shannon's information theory, we define the mutual informa- 
tion between two quantum systems as follows: 

Definition 2. The information in subsystem B about subsystem A (and 
vice versa) is 

I{A-B)= sup I (a; 13). (15) 

MaMb 

The classical equality (fT3j) then turns into an inequality: 
Theorem 3. 

I(A;B)<H(A)-H(A\B). (16) 

Proof. From the entropy defect bound [Levitin 1969|IHolevo 1973j it follows 
that for any A4b 

I (A; (3) = sup I(a; (3) < H(A) - H(A\p), (17) 

M A 

where equality holds iff all p(A\(3 = b) commute and M.a is a von Neumann 
measurement in the basis where all p(A\(3 = b) are diagonal. 
From ffT7|) . 

I(A; B) — sup l{a,0) 

MaMb 

< sup[H(A) - H{A\[3)} = H(a) - inf H(A\(3) 

M B Mb 

= H(A) — H(A\B). m 
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The proposed measures of information and conditional entropy turn out to 
be useful in the analysis of correlated (in particular, entangled) quantum 
systems, in place of their classical counterparts. 
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